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ABSTRACT: Although robotic solutions have been making significant contributions to fabrication environments, 
implementations in the construction are rare. It seems a disconnect between the industries exists where in 
construction the high number of non-uniform work tasks, the wide assortment of types and shapes of building 
materials and elements, and the presence of human workers creating safety hazards make the deployment of rather 
rigid robotic manipulators on construction sites much more complex than in production-like work environments. 
To advance construction with robotic solutions, it could prove beneficial to make each sector aware of the barriers 
that exist, and likewise, introduce a physical space for joint experimentation with state-of-the-art technologies 
from both fields. One way of alleviating this issue is to connect the sectors by providing hands-on education and 
research experiences, defined hereby as Learning Factory for Construction (LFC). This paper presents a scaled- 
down version of a LFC that has a robotic manipulator perform fully-automated and precise assembly, 
deconstruction, and reuse tasks of modular construction elements, whereas the elements are tracked with fiducial 
markers according to a known building information model and schedule. Furthermore, the FLC continuously 
gathers and analyzes data for performance, measures successful completions, assembly times, and potential 
quality defects. This project involved Masters level students with domain expertise from architectural, civil, and 
mechanical engineering in a cross-disciplinary and collaborative learning exercise of building a working 
prototype within a semester-long study project. Beyond the core tasks of the digital design and robotic application, 
the group developed theoretical concepts and limitations for more holistic views on circular economy, lean 
production, on- and off-site logistics, modularization, and construction safety, just as expected from a LFC. It is 
anticipated that the next generation of professionals working in the built environment and intending to solve some 
of the larger and more complex societal problems will require both the technical and communication skills that a 
LFC can stimulate. Therefore, LFC is expected to become an important component of active learning environments. 
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1. INTRODUCTION 


For the past decades there has been an increased interest in robotic technology in construction applications. 
Economic projections foresee a prospering field and actual widespread usage in practice, requiring new policies 
and rules across the impacted industries (EC, 2022). However, many challenges still present themselves regarding 
robots in construction. Simple tasks that prove easy to execute for humans, prove extremely difficult for robotic 
manipulators due to a lack of perception and cognitive abilities. The size and weight of robots in industrial work 
environments, often tackling singular and highly repetitive tasks, does not fit the challenging, complex, and highly 
dynamic work environment that exists in construction sites. Yet, finding the necessary functionality and usability 
are a few of the additional barriers that exist and prevent robots from mainstream implementation. Despite some 
recent and rather serious interest from the industry, robotic applications in construction have stayed limited to 
niche research or exploration projects. Automated and robotic brick laying machines (Usmanov et al., 2017; Ravi 
et al. 2021) and additive manufacturing are some examples (Teizer et al., 2016). 


To enable the use of robotics, suitable methods to assist the robots are necessary to consider. Yet, they are difficult 
to develop as construction touches a multi-disciplinary field that makes it challenging to find acceptable solutions. 
A few somewhat isolated disciplines (and stakeholders) are: design (architects/planners), construction (civil 
engineers), machinery (mechanical engineers), and systems and processes (industrial engineers). While innovation 
in any field, like in construction, calls for lifting the boundaries between these domains, a further major aspect to 
consider before introducing robotic applications in construction is to maintain a high level of trust, productivity, 
and safety in new technologies (EC, 2022). 


Change to fabrication environments came over decades, with fully-automated solutions replacing isolated and 
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highly repetitive work tasks humans would not endure. The typical construction work environments yet may 
demand a similar time frame and even more. For example, active human-robot collaborations are supposed to 
solve the sector’s rather complex and interconnected work tasks. The involvement of multi-trades’ expertise and 
the manifold types of product or material specifications constitute a few of the other but plentiful technical 
challenges that semi- or fully-automated robotic solutions are envisioned to solve before decision-makers would 
buy into them for final field use (Slaughter, E.S., 1998; Goodrum & Haas, 2016) 


Yet, the effects that a transition to robotic labor would have on construction can include improvements to 
construction industry-wide problems, including but not limited to achieving higher productivity and better safety 
and health performances. As such, prioritization of human time and purpose of life and health, and ease of system 
installation and maintenance, to name only two criteria, reflect the current construction industry’s efforts towards 
digitalization, automation and robotization (Yamamoto, 2020). 


The concept of a Learning Factory (LF) is not new, and yet they hardly exist for construction purposes. Teizer and 
Chronopoulos (2022) expressed that a Learning Factory for Construction (LFC) can provide a useful active 
collaborative working environment for engineers that are interested in exploring prototypical solutions that have 
the potential to solve known industry problems. In their articulated vision, a LFC provides the explorative 
collaboration space to (a) detect the organizational barriers that prevent innovation, (b) allow objective and scope 
definitions by understanding the technical limitations in existing work processes, and (c) create prototypical hard- 
and software solutions that can be tested on small but at realistic scale and with little risk of losing large investments. 
Gaining knowledge in a LFC first is required to later adapt solutions to a larger workspace and with increased 
autonomy. And yet, students that participate in a LFC should have fun, like Teizer et al. (2020) and Wolf et al. 
(2022) found out when observing construction apprentices that played serious games for construction safety. 


As the widespread application of robotics in manufacturing industries has significantly improved productivity and 
efficiency, there has been significant research interest in construction robotics. Besides reducing project delivery 
delay, construction robotics can benefit the workers by assisting them with non-ergonomic tasks (e.g., lifting 
weights) and taking over dangerous activities (e.g., demolition). However, the implementation of construction 
robotics heavily relies on manual input from task to task due to the complicated nature of construction activities. 
For instance, the difference between as-built and as-designed models during the construction stage can be 
challenging for preprogrammed construction robots to understand the changing environment at the construction 
site. Only in combination with a higher level of digitalization construction robotics can it be effectively 
implemented for automated or semi-automated construction. Emergent methods and technologies, such as BIM 
and vision-based object recognition, collaborate with construction robotics to complete the workflow of automated 
construction. Such collaboration requires various fields of engineering to understand all involved technology and 
the interplays between these technologies. Figure 1 integrates many of the currently existing digital technologies 
and how they relate to each other. Highlighted in grey background color are those that are part of this LFC. 
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Fig. 1: Overview of relation between digital technologies; modified, originally from EC (2021), 


The goal of this paper is to demonstrate the viability of the integration of a robotic manipulator into a LFC, 
displaying the advantages of using modular components in the context of autonomous construction in a circular 
economy. The following sections first review the background, then introduce the developed LFC, a scaled-down 
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version of a building construction site, and finally demonstrate its capabilities in a case study where a robotic 
manipulator handles modular elements for building assembly and reuse under some of the typical real constraints 
that exist in the construction supply chain and in a circular economy. 


2. BACKGROUND 


Several existing cases have shown that assembly processes using robotic manipulators are favorable. Wang et al. 
(2020a) stated that robotic construction was both faster and more accurate compared to conventional manual 
methods by construction workers alone. However, it was stated that robotic solutions were limited to being either 
conducted in non-complex work environments or limited to handling specified objects. In other words, robotic 
solutions still required some aspects of human labor to complete complex activities. These limitations pose some 
of the biggest challenges to overcome. 


The recently-completed research project HEPHAESTUS proved successful in installing curtain wall modules 
using a large-scale cable-driven robot alongside a robotic manipulator, but many improvements were left to be 
implemented (Iturralde et al., 2020). 


Using robotic manipulators for construction has shown to be easier executed when introducing the modular and 
parametric design in assemblies of complex constructions. Research on modular design for robotic construction 
showed that it is possible, using modular components, to verify the design and construction process through 
simulations (Sun et al., 2022). 


Using timber panels, which are identified by computer vision and machine-readable QR codes, has proved to make 
it possible for a robot arm to do insertions of panels to create simple assemblies (Rogeau et al., 2020). In addition, 
a robot arm using standardized timber was able to construct complicated structures with high precision. Results 
from Leng et al. (2020) showed the benefits and possibilities of utilizing standardized materials, with precise 
parameters being a key factor. A similar research with timber addressed the issues of wood being a natural and 
imprecise material which complicates handling, highlighting the issues of production tolerances (Hasan et al., 
2019). 


The notion of having robots build from a digital model has been investigated in several papers, with a focus on 
exporting Building Information Modeling (BIM) to a robot from an as-designed model or importing the physical 
as-built model for guidance purposes. Likewise, using a Digital Twin for Construction (Sacks et al., 2020), the 
process of having a robot build from a BIM and updating the as-built model using sensor data was validated (Wang 
et al., 2020b). At the moment, software packages are being developed to help link BIM-based design with robot 
control which could ease the process of future digital-to-physical model building (Yang et al., 2019). Slebicka et 
al. (2021) placed an important vision for Fabrication Information Modeling (FIM) that intends to close the gap 
between BIM and Digital Fabrication that, at some point in time, will heavily depend on automation and robotics. 


While only a small amount of the above-mentioned research addresses the interaction between autonomous robots 
and human workers, human-robot interaction proves to be detrimental when considering on-site safety (Wu et al. 
2020). With the prospect of robots in construction, it is recommended to also investigate their social impact since 
the potential changes to workplaces will require workers to acquire additional skills, competencies and 
responsibilities (Karl et al., 2018). A proposed method of tackling safety is to introduce the concept of an LF, which 
emphasizes hands-on experience. LFs offer a high potential to improve education, training, and research in a 
controlled environment (Abele et al., 2017). 


Gharbia et al. (2019) concluded that rapid prototyping assisted in the creation of robotic solutions. In this context, 
introducing a robot manipulator into a LFC would educate on, and increase the awareness of the capabilities of 
autonomous robots in a construction work environment and help involved project stakeholders (engineers and 
workers) adapt increasingly advanced technologies to a construction site. 


Related to this effort, robots of different sorts have already been introduced to various Learning Factories in 
relation to Industry 4.0. Several researchers have included robotic arms in their own specialized Learning Factories, 
albeit with a focus on manufacturing and assembly (Matt et al., 2014; Kaménzy et al., 2018; Nardello et al., 2017). 


As with Industry 4.0, the recent technological advances will gradually replace the roles of humans in construction, 
in what is coined Construction 4.0 (Sawhney et al., 2020). However, the challenge of integrating a robot into a 
LFC environment with the purpose of improving construction processes is yet to be investigated. 
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3. ROBOTIC MANIPULATOR IN A LEARNING FACTORY FOR CONSTRUCTION 


The first part of this section briefly explains the relevant backgrounds of the research methods employed in this 
work of Masters-level students in a semester-long study project that utilizes the LFC at the Technical University 
of Denmark. Next, the hard- and software components of the LFC are introduced. Experiments and results follow 
with a discussion summarizing the lessons learned at the end. 


3.1 Introduction to components 
3.1.1 Learning factory for construction 


LF has proven to be an effective way to provide active hands-on learning (Abele et al., 2017). LF typically 
reproduces or simulates a production environment, allowing participants to gain practical knowledge and skills in 
a controlled setting. In addition, LF also provides a platform for researchers to investigate and improve processes 
and workflows. In this case, our LFC is meant for university students but can also involve apprentices or full-time 
professionals, like workers, technicians, and engineers from the construction industry. The purpose of our LFC is 
to provide the physical space that facilitates education and research on automation and robotics in construction. 


3.1.2 Robotic manipulator 


A robotic manipulator performs tasks as a human arm (Matt et al., 2014). In our case, the robot mimics a mobile 
or tower crane on a construction site. Our LFC consists of multiple modules that include robotic elements, of which 
only the robotic manipulator URSe, its mounted camera and gripper, and a computer will be explained in the 
further text. Details of the other existing components of our LFC can be found in Teizer and Chronopoulos (2022). 
While these eventually will be connected to each other, this paper introduces the part of the LFC that simulates the 
process of three steps in automated modular construction: automated assembly, disassembly, and reuse according 
to a BIM-based building design. The URSe is made of several interconnected segments, has joints, and one end- 
effector, allowing it to make rotary and linear movements. The end-effector performs assigned tasks at any position 
within the spatial coordinates of the robotic arm. The learning factory uses URSe as the robotic arm and mounted 
gripper as the end-effector so that it can grab, lift and place any given components at the assigned positions. It has 
six degrees of freedom (x, y, z, roll, pitch, yaw), whose value can be changed so that the gripper mounted at the 
end of the arm can be moved to desired position and orientation. There exist three types of movements, moveJ (the 
robot moves each joint independently), moveL (the robot moves in a straight line), and moveP (the robot moves 
following the designed path). 


3.1.3 Building information model and construction schedule 


BIM is a comprehensive and collaborative method across the whole building life cycle (Oraee et al., 2017). Yet, it 
has less been used in combination with automation and robotics than other applications. Our LFC uses 
commercially-available BIM software for the manual design of a fictive modular building project and, likewise, 
is the sequence of constructing the modular elements planned digitally. While this may imply a detailed 
construction schedule comprised of the precise timing and dependencies of the construction tasks, only the Work 
Breakdown Structure (WBS) is needed. The BIM software is also used for visualization purposes. Otherwise, the 
IFC format contains geometry and position information for each of the modular elements and the task sequence. 


3.1.4 Building materials 


The building is constructed with standardized physical models using the URSe. The pieces are made from 
lightweight plastic and come in several shapes. 


3.1.5 Object detection 


In order for the robot to handle the modular elements, object detection and recognition with final localization is 
required. Object detection is made possible by computer vision that identifies and localizes the modular elements 
of next interest within the video frame capture. There exist two main approaches for object detection, traditional 
(e.g., rule-based, handcrafted features) and deep learning-based approaches. Out LFC integrates traditional object 
detection algorithms for construction sites and modular component detection. 


3.1.6 |Human-robot interaction 


Human-robot interaction happens only twice in this part of the LFC: first, to place new modular elements in the 
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arrival area to the simulated construction site that is within range of the robot arm, and second, when the building 
owner makes a choice of selecting a building design and floorplan. Otherwise, the developed module of the LFC 
operates fully autonomously, as explained in the following. 


3.2 Methods 


The framework of the LFC is shown in Figure 2. It comprises two parts. The first part is a remote control, where 
students in the role of an architect or civil engineer upload their IFC file to the computer. Note that the modular 
building designs of the students allow some variation but still follow the material specifications and parameters 
that were given to them beforehand. The computer extracts the geometry and position information of each modular 
building component and determines the construction sequence. The spatial and temporal information of 
construction is translated into robot commands and then sent to the robot for controlled execution. The manipulator 
first scans the entire site to locate and map the coordinates of the material pick-up, the temporary depot, and the 
construction zones. When ready, the robot finally receives the building commands to start the construction process. 
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Fig. 2: Learning Factory for Construction (LFC) at the Technical University of Denmark: Construction site 
module. 


The hardware and software requirements and descriptions for the learning factory are listed in Tables 1 and 2. The 
BIM translation and robotic remote control are implemented in a Python environment due to its simplicity and 
extensive library support. 


Table 1: Hardware in the construction site module of DTU’s LFC. 


Equipment Description 


Robotic manipulator URSe for grabbing, lifting, and placing building components 
Camera on end-effector OnRobot RGBD camera for object detection and as-performed data collection 


Computer Processing BIM files, translating commands, receiving and processing data, control 


Table 2: Libraries and software for the learning factory 


Library and applications Version Description 

URX 2.0.1 URSe remote control and program execution 
IfcOpenShell 1.6.1 IFC file translation and querying 

OpenCV 20.10.22 Visual detection ad recognition 

BlenderIFC IFC file editing and viewing 


For reliable object detection, the camera uses fiducial marks attached to the construction site and building 
components to recognize the different objects, as shown in Figure 3. 
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Fig. 3: Examples of (a) a zone and (a) modular building elements, all marked with different fiducial marks for 
object recognition. Note. The elements are further detected by shape and color. 


3.3 Implementation and preliminary results 


The preliminary implementation of this component of DTU’s LFC is shown in Figure 4. The layout and setup 
follow the concept mentioned earlier. For a simplified demonstration, a basic two-story building consisting of 5 
IFC elements is designed, shown in Figure 5. The five components are positioned at distinct heights so that the 
algorithm can easily sort the order of construction. Figure 6 shows the simplified modular building elements to 
which each unique fiducial markers are attached. Existing computational algorithms later detect and recognize the 
fiducial marker when it is within the field-of-view of the mounted camera on the end-effector. 
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Fig. 4: Experimental setup of LFC. Fig. 5: IFC model of the 2-story modular building project 


(by (c) 
Fig. 6: Modular components: (a) foundation, (b) wall, and (c) floor. 


Figure 7 illustrates the construction process. After the students load their IFC file, the computer extracts the spatial 
information of building components and determines the construction sequence. The corresponding list of the 
modular elements and the building sequence is shown in Table 3. The robot registers the coordinate of the 
construction sites by using the mounted camera to detect the fiducial markers of the zones. After the coordinate 
system is registered, the robots start to detect, grab, lift, and place the modular construction elements iteratively 
until the last component is assembled onto the building. Reversely, the disassembly process can also be achieved. 
All modular elements are taken apart and placed in a temporary storage zone (called depot). Next, human 
interference is needed if the next phase of the building lifecycle is of interest to the student. The student can choose 
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to select an alternative building design, upload it, and the new building process can start again. Note, whenever 
possible, the robot reuses parts of the modular building elements lying in the depot. Once the second building is 
completed, typically, the LFC experience stops. 


1. Load the IFC file and find 
elements 


2. Sort the assembly order of 
the elements 


3. Register the pick-up, 
depot, and construction zone 


7 
Repurpose the disassembled elements 


5. Grab, pick up, and place 
building elements until all are 
assembled 


6. Disassemble the building 
elements if required 


4. Identify the building 
elements 


Fig. 7: LFC-workflow of the robotic manipulator module: First the design, then the robotic assembly and 
disassembly, optional: re-design and -use. 


Table 3: List of the modular building elements and example of a building sequence by the ascending z coordinate 


Number Elements Coordinates Sequence 
1 'Foundation:297060' (x1,y1,z1) 1 

2 'Floor:301328' (x2,y2,z2) 

3 'Floor:301575' (x3,y3,z3) 5 

4 "Wall:3048 10" (x4,y4,z4) 2 

5 "Wall:305546' (x5,y5,z5) 4 


As the algorithm is sorting the construction sequence by a bottom-up approach, it may only be viable for modular 
building elements with simplistic spatial relations. The construction order can be determined using a predefined 
construction schedule in 4D BIM. For each IfcElement, IfcTask and IfcRelAssignsToControl are attributed so that 
the algorithm can understand the predecessor of each step and validate the correct order during the construction 
stage. 


While, iterative occurred during the system’s development, demonstrating that the entire workflow was tested 5 
times in front of a small audience from industry and academia. Although no strict scientific verification and 
validation methods were ready at the end of the semester project, the students were able to run the system two 
times successfully from start to end. Twice the students assisted by snapping an element (one floor and one wall 
element, in separate tests) into place with a very slight push of an index finger. Once the robot stopped after 
assembly the first design. The reason is still unknown. Yet, runtime data from the system was recorded during all 
test runs and is being processed at this time (and will be implemented in the final version of this paper). 


Fig. 8: Impressions from final demonstration exercises: Robot manipulator completing the fully-automated (a) 
assembly of the first modular building design, (b) disassembly and temporary storage, and (c) re-use of modular 
elements for assembly of second building design (manual selection, after disassemble). 
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4. CONCLUSIONS AND OUTLOOK 


This work is the result of a semester-long study project that exposed four Masters-level students, one in 
architectural, one in civil, and two in mechanical engineering, to backgrounds that they had not learned before. 
For example, both the architectural and civil engineering student had no previous experiences with the field of 
automation and robotics, and likewise had mechanical engineering students neither a background in design or 
planning with 4D BIM nor any expertise in modular construction. The developed concept of a LFC has been 
partially validated, as the robot manipulator was able to follow a digital design and sequence to erect, disassemble, 
and rebuild a small-scale building while applying constraints that exist in a circular economy, for example, making 
as much use as possible of reusing building material. However, as observed, the limited project time that was given 
to the students restricted their curiosity in exploring additional research domains, for example, planning for 
alternatives, generative costing, digital twinning, and testing usability. In the future, a focus on qualitative and 
quantitative assessment methods must be set to evaluate both the students’ and LFC’s performances. Yet, the 
students’ claimed new knowledge by applying their own expertise and discovering other fields. Furthermore, the 
experienced hands-on experiences with respect to realistic and still basic implementations of information modeling, 
computational coding, automation, and robotics, strengthened their learning. It is envisioned that the construction 
industry will benefit from students with such skill sets that a LFC is able to develop, share, or enhance. Yet, scaling 
up the developed concept of a LFC could yield future insights how digital building design can guide real-life 
automation and robotic applications in construction. 
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